1. Field of Invention
This invention relates to 3D point of gaze apparatus and more particularly relates to an apparatus system and method for mobile, low-cost, head-mounted, 3D point of gaze estimation.
2. Description of Related Art
Eye gaze based interaction has many useful applications in human-machine interfaces, assistive technologies, and multimodal systems. Traditional input methods, such as the keyboard and mouse, are not practical in many situations and can be ineffective for some users with physical impairments. Knowledge of a user's point of gaze (PoG) can be a powerful data modality in intelligent systems by facilitating intuitive control, perception of user intent, and enhanced interactive experiences.
Gaze tracking devices have proven to be extremely beneficial to impaired users. In one case study presented (V. Pasian, F. Corno, I. Signorile, and L. Farinetti. The Impact of Gaze Controlled Technology on Quality of Life. In Gaze Interaction and Applications of Eye Tracking: Advances in Assistive Technologies, chapter 6, pages 48-54. IGI Global, 2012.) sixteen amyotrophic lateral sclerosis (ALS) patients with severe motor impairments (loss of mobility, unable to speak, etc.) were introduced to eye tracking devices during a 1-2 week period. The patients were assessed by a psychologist during an initial meeting in order to evaluate their general quality of life. Eye tracking devices and proper training, as well as access to a speech and language therapist and a computer engineer, were provided for the duration of the study. Patients completed questionnaires related to their experiences with the equipment several times during the study. Several patients reported a clear positive impact on their quality of life during the study, resulting from the enhanced communication facilitated by the eye tracking devices over other non-gaze based assistive devices.
While the utility of gaze interaction in a variety of applications has been demonstrated, the availability of the technology has been a limiting factor in more widespread use. Due to the relatively high monetary cost and proprietary nature associated with commercial eye tracking equipment and software, several low-cost solutions have been developed using inexpensive on-the-shelf components. Many of these designs have been made publicly available through the open source community. The openEyes project (D. Li, J. Babcock, and D. J. Parkhurst. openEyes: a low-cost head-mounted eye-tracking solution. In Proceedings of the 2006 symposium on Eye tracking research & applications—ETRA '06, page 95, New York, N.Y., USA, 2006. ACM Press.) presents a low-cost head-mounted eye tracker that uses a pair of inexpensive IEEE-1394 cameras to capture images of both the eye and scene. This hardware device, coupled with the open source Starburst algorithm, facilitates estimation of the user PoG in the 2D scene image. A similar open source project, the EyeWriter, provides detailed build instructions for creating a head-mounted eye tracker from a modified Playstation Eye USB camera. The project was designed to enable digital drawing by eye gaze control for artists with ALS while using the device with the accompanying open source software. Interestingly, in J. San Agustin, H. Skovsgaard, J. P. Hansen, and D. W. Hansen. Low-cost gaze interaction: ready to deliver the promises. In Proceedings of the 27th international conference extended abstracts on Human factors in computing systems—CHI EA '09, page 4453, New York, N.Y., USA, 2009. ACM Press., the effectiveness of a low-cost eye tracker is shown to be comparable to that of commercial devices for target acquisition and eye-typing activities.
The head-mounted eye gaze systems mentioned above facilitate effective interactive experiences with some limiting constraints. In general, these solutions are designed for interaction with fixed computer displays or 2D scene images. These types of systems provide a 2D PoG, which does not directly translate into the 3D world. An accurate estimate of the 3D user PoG can be especially useful in mobile applications, human-robot interaction, and in designing intelligent assistive environments. Knowledge of the 3D PoG within an environment can be used to detect user attention and intention to interact, leading to multimodal attentive systems able to adapt to the user state.
Some mobile 3D PoG tracking systems have been proposed in literature. For example, a head-mounted multi-camera system has been presented that estimates the 3D PoG by computing the intersection of the optical axis of both eyes. This approach gives the 3D PoG relative to the user's frame of reference, but does not provide a mapping of this point to the environment in which the user is present. A similar stereo camera approach is presented in K. Takemura, Y. Kohashi, T. Suenaga, J. Takamatsu, and T. Ogasawara. Estimating 3D point-of-regard and visualizing gaze trajectories under natural head movements. In Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications—ETRA '10, volume 1, page 157, New York, N.Y., USA, 2010. ACM Press., which also includes a forward-facing scene camera for mapping of the 3D PoG to scene coordinates. While multi-camera approaches such as these provide a 3D PoG, their use is limited by increased uncertainty at increasing PoG depths. Another limiting factor is the scene camera, which is generally a standard 2D camera that does not provide any 3D information of the environment itself.